Skip to content

refactor(engine): type capture failures#2161

Open
jrusso1020 wants to merge 2 commits into
mainfrom
07-10-refactor_engine_type_capture_failures
Open

refactor(engine): type capture failures#2161
jrusso1020 wants to merge 2 commits into
mainfrom
07-10-refactor_engine_type_capture_failures

Conversation

@jrusso1020

@jrusso1020 jrusso1020 commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

What

Replace message-parsed capture retry policy with typed CaptureFailure values and peer abort.

Why

Stringified worker errors lost causes and let sibling workers continue after fatal authoring failures.

How

Classify cancellation, browser, protocol, memory, verification, authoring, and IO failures; preserve diagnostics and abort peers on the first fatal kind.

Test plan

  • capture classification and parallel peer-abort tests; producer capture-stage tests
  • Stack-wide lint, format, build, typecheck, and relevant integration gates

jrusso1020 commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_manage_child_processes branch from 5e82fec to fc61671 Compare July 13, 2026 23:08
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_type_capture_failures branch from 131d9c1 to 67f5483 Compare July 13, 2026 23:08
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_manage_child_processes branch from fc61671 to b683ddb Compare July 14, 2026 00:03
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_type_capture_failures branch 2 times, most recently from 7935b9d to bd29456 Compare July 14, 2026 03:20
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_manage_child_processes branch from b683ddb to 4e86382 Compare July 14, 2026 03:20
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_type_capture_failures branch from bd29456 to e10f87a Compare July 14, 2026 17:03
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_manage_child_processes branch from 1a7d917 to 7dd4686 Compare July 14, 2026 17:11
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_type_capture_failures branch from e10f87a to 9e02fbc Compare July 14, 2026 17:11
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_manage_child_processes branch from 7dd4686 to 8733675 Compare July 14, 2026 17:35
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_type_capture_failures branch 3 times, most recently from 533dc78 to 5d9ca34 Compare July 14, 2026 18:06
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_manage_child_processes branch 2 times, most recently from 0c42de6 to f095fb4 Compare July 14, 2026 19:37
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_type_capture_failures branch from 5d9ca34 to 2d9a934 Compare July 14, 2026 19:37
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_manage_child_processes branch from f095fb4 to eda0bb1 Compare July 14, 2026 19:42
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_type_capture_failures branch from 2d9a934 to d564f84 Compare July 14, 2026 19:42
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_manage_child_processes branch from eda0bb1 to c0bd5b9 Compare July 14, 2026 19:47
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_type_capture_failures branch from d564f84 to 8c275ea Compare July 14, 2026 19:47
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_manage_child_processes branch from c0bd5b9 to 571341a Compare July 14, 2026 19:55
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_type_capture_failures branch from 8c275ea to 01d355d Compare July 14, 2026 19:55
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_manage_child_processes branch from 571341a to 606e05b Compare July 14, 2026 23:29
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_type_capture_failures branch from 01d355d to 26d4d2e Compare July 14, 2026 23:29
@jrusso1020
jrusso1020 force-pushed the 07-10-refactor_engine_manage_child_processes branch from 606e05b to 6cf4221 Compare July 14, 2026 23:41
Comment thread packages/engine/src/services/captureFailure.ts Fixed

@miguel-heygen miguel-heygen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The typed failure boundary is a strong improvement overall: cancellation is authoritative, worker diagnostics stay immutable and attached to the representative failure, and the first fatal worker now stops sibling capture without hiding cleanup. The linear same-line I/O scan and 100k-character adversarial test also close the CodeQL finding without broadening classification.

One retry regression remains:

  • [P1] Preserve retryability for the drawElement worker watchdog timeoutpackages/engine/src/services/captureFailure.ts:57-64 enumerates protocol timeout strings but omits the engine's own drawElement worker encode timed out (frame N) failure. That error is produced by the 30-second lost-message/OOM-killed-worker watchdog at packages/engine/src/services/drawElementService.ts:674-684 (and the batch sibling at :901-906), then propagates through captureFrameRange's awaited encodeResult. On this exact head, classifyCaptureFailure(new Error("drawElement worker encode timed out (frame 42)")) returns authoring, so parallelCoordinator.ts:594-598 treats it as fatal and aborts peers, while renderOrchestrator.ts:817-821 refuses the worker-halving retry. Before this PR, the broad timed out recovery matcher covered this concrete failure. Please classify this exact watchdog failure as non-fatal/retryable and pin both classification and recovery behavior with a regression test.

Exact head reviewed: 9ad957786d14040f72b87ac2f72ac59a028e6f90. All exact-head required checks are green; this is a code-contract blocker rather than a CI failure.

Verdict: REQUEST CHANGES
Reasoning: The typed model and peer-abort mechanism are sound, but a real engine-generated worker timeout regresses from retryable to fatal, causing avoidable render failures instead of the prior recovery path.

— Magi

@jrusso1020

Copy link
Copy Markdown
Collaborator Author

Fixed Magi’s P1 on exact head 5da9f7ab3d53d7b5b95f216d63532f20cb6036dd.

  • Added a narrowly scoped drawElement worker encode timed out (frame N) protocol-timeout classification, preserving non-fatal peer behavior and Producer’s worker-halving retry.
  • Added direct Engine classification coverage and a wrapped [Parallel] Capture failed Producer recovery regression.
  • Validation: Engine classification 13/13; full Producer render-orchestrator 142/142; full Engine 1,131 passed / 3 skipped; Engine and Producer typechecks; lint/format/tracked-artifact/fallow hooks; package-subpath sync; full workspace build on exact restacked top 4deb4510c18b79c95b8c5dc1a64420257c437a03.
  • PRs refactor(engine): type capture failures #2161refactor(repo): resolve changed-code audit #2174 were restacked and republished. Exact head remains on live main c268f5ba8 with behind_by: 0 and merge base exactly live main.

Fresh exact-head CI is running. This is a substantive head change and requires a fresh exact-head review after all required checks are green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants